On nonnegative garrote estimator in a linear regression model

Abstract: Subset selection regression is a frequently used statistical method. It waives some of the predictor variables and the prediction equation is based on the remaining set of variables. Subset selection is simple and it clearly reduces the variance. An other method for reducing the variance is ridge regression. Usually, subset selection is not as accurate as ridge. The problems with ridge regression are for example: 1) it is not scale invariant 2) it does not give a simple equation. We need an intermediate method which selects subsets, is stable and gains its accuracy by selective shrinking. Breiman (1995) proposed a new method, called the nonnegative (nn) garrote. In this lecture, in a linear regression model, we consider the nonnegative garrote estimator of the coefficients as introduced by Breiman (1995). This estimator shrinks the least square estimator by a parameter λ in the orthogonal case. In an especial case of λ, we prove the nn-garrote estimator is consistent and its MSE converges to zero. We also obtain the rate of convergence.